Performance and Convergence of Multi-user Online Learning

We study the problem of allocating multiple users to a set of wireless channels in a decentralized manner when the channel qualities are time-varying and unknown to the users, and accessing the same channel by multiple users leads to reduced quality (e.g., data rates) received by the users due to interference. In such a setting the users not only need to learn the inherent channel quality and at the same time the best allocations of users to channels so at to maximize the social welfare. Assuming that the users adopt a certain online learning algorithm, we investigate under what conditions the socially optimal allocation is achievable. In particular we examine the effect of different levels of knowledge the users may have and the amount of communications and cooperation. The general conclusion is that when the cooperation of users decreases and the uncertainty about channel payoffs increases it becomes harder to achieve the socially optimal allocation. Specifically, we consider three cases. In the first case, channel rates are generated by an iid process. The users do not know this process or the interference function, and there is no information exchange among users. We show that by using a randomized learning algorithm users converge to the pure Nash equilibria of an equivalent congestion game. In the second case, a user is assumed to know the total number of users, and the number of users on the channel it is using. We show that a sample-mean based algorithm can achieve the socially optimal allocation with a sub-linear regret in time. In the third case, we show that if the channel rates are constant but unknown, if a user knows the total number of users, then the socially optimal allocation is achieved in finite time with a randomization learning algorithm.